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AUTOMATIC IMAGE QUALITY EVALUATION AND CORRECTION 
TECHNIQUE FOR DIGITIZED AND THRESHOLDED DOCUMENT 
IMAGES 
FDCLD OF THE INVENTION 

The invention relates to a process and system for automating the 
image quality inspection and correction for scanned documents which previously 
required a human operator. The process and system for each and every scanned 
and thresholded image performs an automatic evaluation through a binary image 
quality detection system which generates an image noise index indicative of the 
amount of image artifacts or image loss. When a poor quality scanned page is 
detected, e.g., too much speckle noise is present, the gray scale image is retrieved 
or the image is rescanned in gray scale. The gray scale image then automatically 
undergoes an image quality correction process to produce a clean, readable binary 
image. 

BACKGROUND OF THE INVENTION 

In a production scanning environment, the digital output of a 
scanned paper document is often represented and stored in binary (Black and 
White) form because of greater efficiency in storage and transmission, particularly 
for textual images. The capture of a binary image requires an image thresholding 
process installed in a scanner, which converts digital gray scale signals captured 
by a CCD sensor (usually, 8 bits per pixel) into binary signals (1 bit per pixel). 
Since an image thresholding is an image data reduction process, it often results in 
unwanted image artifacts or image information loss, such as speckle noises in the 
document background or loss of low contrast characters. In the prior art, there are 
numerous adaptive thresholding techniques for producing an optimal quality 
binary image. However, an adaptive thresholding technique has yet to be found 
that automatically produces a clean, readable binary image for every captured gray 
scale image in a batch of paper documents due to variations of paper background 
and content. As a result, the production document scanning often requires 
intensive labor in the form of visual image quality inspection warranting that 
every captured binary image is readable. 
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Clearly, there is a need for an improved scanning system and 
process that is capable of producing a clean, readable binary image of text or the 
like without the need for a visual image quality inspection. Ideally, such a system 
and process would be sufficiently compatible with presently available scanning 
5 components to allow the use of the system on scanners presently in use, and to 
minimize the need for the design and manufacture of new components. 

SUMMARY OF THE INVENTION 
The invention is an image processing system and process that 
automates the image quality inspection and correction previously performed by a 

10 human operator. The image quality of every scanned and thresholded image is 

automatically evaluated by a binary image quality detection system that generates 
an image noise index for every image. When the image noise index indicates a 
poor image, such as too much speckle noise in background, the gray scale image 
is retrieved or the document is re-scanned in gray scale. An image quality 

1 5 correction process is automatically applied to the gray scale image to produce a 
corrected, clean, and readable binary image. 

It is an object of the present invention to automatically index image 
noise level for a thresholded image which is generated by a multi-windowing 
technique for thresholding an image using local image properties in real time. For 

20 every thresholded image having high image noise index, an image quality 
correction scheme is applied to the problematic document and produces an 
optimal binary image quality in post processing. The design of the system allows 
performing high speed image quality inspection while retaining optimal image 
quality output without operational interruption in high speed document scanning 

25 environment. 

The image processing system comprises two image processing 
components: a binary image quality inspection component for evaluation of 
image noise level and a multiple adaptive thresholding image processing 
component for correction of image quality. The binary image quality inspection 
30 component evaluates the noise level of a thresholded image and takes the noise 
level as an index of image quality. When the noise level is greater than a 
specified threshold the image is considered as poor image quality. For every poor 
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thresholded image, its original gray scale image is retrieved if available, otherwise 
the original document is recaptured in gray scale. The correction of image quality 
is made by feeding the gray scale image into a multiple adaptive thresholding 
processor. The multiple adaptive thresholding technique comprises two iterative 
5 image processing steps: adaptive image thresholding and binary image analysis. 
The iteration proceeds until an optimal binary image is found. An optimal binary 
image means an image having clean background and readable text. 

The invention and its objects and advantages will become more 
apparent in the detailed description of the preferred embodiment presented below. 
1 0 BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows the block diagram of the image processing for 
automatic image quality inspection and correction in a scanning system. 

Figure 2 shows the block diagram of the Multi-windowing 
technique for thresholding an image using local image properties. 
1 5 Figure 2A illustrates the image quality of a thresholded image 

controlled by the input threshold parameters (contrast threshold (CT), intensity 
threshold (IT) ). 

Figure 2B illustrates the relationship of the two input parameters 
(contrast threshold, intensity threshold) and the sensitivity of thresholding. 
20 Figure 3 shows the image processing steps of automatic image 

quality inspection for a bitone image. 

Figure 3 A shows the neighborhood operation of binary image 

dilation. 

Figure 3 B is an example of noise dot structures before image 
25 dilation in a binary image. 

Figure 3C is the noise dot structures of Figure 3B after image 

dilation. 

Figure 3D is the image processing block diagram of noise feature 

extraction. 

30 Figure 3E is an example of noise feature calculation in a binary 

image. 
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Figure 4 shows the block diagram of the image processing for 
automatic image quality correction. 

Figure 4A shows the block diagram of the image processing for the 
multiple adaptive image thresholding. 
5 Figure 4B is the table containing the calculated noise levels for 

different thresholded images produced by various input parameters (contrast 
threshold (GT), intensity threshold (IT)). 

Figure 4C is a flow chart of noise level analysis in finding the 
optimal contrast threshold and intensity threshold. 
1 0 Figure 4D is the first example of noise index distribution 

(R(GT,IT) ) with constant intensity threshold (IT) at 70 and multiple contrast 
thresholds (GT) varied from 60 to 15. 

Figure 4E is the second example of noise index distribution 
(R(GT,IT)) with constant intensity threshold (IT) at 70 and multiple contrast 
1 5 thresholds (GT) varied from 60 to 1 5. 

Figure 4F is the third example of noise index distribution 
(R(GT,iT) ) with constant intensity threshold (IT) at 70 and multiple contrast 
thresholds (GT) varied from 60 to 15. 

Figure 4G is an example of noise index distribution (R(GT,IT) ) 
20 with constant intensity threshold (IT) at 60 and multiple contrast thresholds (GT) 
varied from 70 to 220. 

Figure 4H is an example of noise index distribution (R(GT,IT) ) 
with constant intensity threshold (IT) at 60 and multiple contrast thresholds (GT) 
varied from 70 to 220. 
25 DETAILED DESCRIPTION OF THE INVENTION 

Figure 1 depicts the block diagram of a digital image processing 
system which automatically inspects the image quality of every thresholded 
image. For a poor thresholded image obtained from the image quality inspection, 
an image quality correction process is then initiated to search for optimal 
30 threshold parameters in order to obtain a legible and clean binary image. The 

image processing system comprises the following steps: (1) a paper document is 
digitally captured in gray scale which typically contain 8 bits per pixel; (2) the 



capture gray scale image (G) is fed through an adaptive image thresholding 
module to convert the gray scale image into a binary image (B); (3) the binary 
image is run through an image quality inspection and an image noise index (EMI) 
is generated from the inspection; (4) the INI value determines the level of image 
5 quality, and the image quality is classified as poor when the INI value is greater 
than a predetermined threshold (ITH); (5) for a low quality image, an image 
quality correction is applied to the gray scale image in order to produce a legible, 
clean binary image. The detail of each image processing module is described in 
the following: 

10 Adaptive Image Threshold 

The multi-windowing technique for thresholding an image using 
local image properties as described in U.S. Patent No. 5,583,659, the disclosure of 
which is hereby incorporated by reference, is applied to convert a gray scale 
image into a binary image. The thresholding technique calculates the threshold 

1 5 value for every pixel in an image based on the local image properties surrounding 
the pixel. The threshold value is then applied to turn the gray value of a pixel into 
bi-level value (black or white). The block diagram of the thresholding technique 
is summarized in Figure 2. The operation of the technique requires a user to 
supply a contrast threshold value (CT) and an intensity threshold value (IT) as 

20 shown in Figure 2A. The image quality of a thresholded image is determined by 
the input of the two threshold values (CT and IT). A thresholded image (CT, IT) 
as shown in Figure 2A means that the image quality of a thresholded image is a 
function of (CT, IT). The image quality of a thresholded image is evaluated by 
two factors: text readability (no image information loss) and smoothness of the 

25 white background (noise free on image background). The CT value controls the 
sensitivity of detecting thin, light text and line arts, and the it value accounts for 
the threshold of a uniform gray region. Assume that the CT value is normalized to 
the values in the range of 0 and 1 00 and the it value is between 0 and 255 as 
shown in Figure 2B. The CT value is often chosen to be sensitive enough for 

30 extracting all image information such as light text, thin lines, . . . etc., and the IT 
value is set to turn light gray uniform region into white background. The 
threshold values at (CT=60, IT=90) as shown in Figure 2B is considered as an 
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appropriate setup to extract all image information details. However, the setup of 
the threshold values at (CT=60, IT=90) may result in too sensitive for some 
documents due to the roughness of document papers and results in generating 
unwanted speckle noises in background of the thresholded images. The following 
5 section describes the method of automatic image noise inspection. 
Automatic Image Noise Inspection 

As mentioned above, the CT value supplied to the thresholding 
technique is always set to be sensitive enough to extract all image information 
details so that the text readability is no longer an issue to be concerned in image 

10 quality evaluation. The evaluation of image quality is simplified to the detection 
of background noises in the thresholded image. 

Figure 3 is the processing steps of detecting noise level for a 
thresholded image. A binary image dilation operation is applied to the binary 
image (B) and generates a new binary image (Bl). Three noise features are 

1 5 measured from the dilated binary image (B 1). The normalized noise area (R) is 
taken as the image noise index (INI) of the binary image (B). The detailed 
description of the processing steps as shown in Figure 3 is stated in the following. 

Assume that black pixel is marked with " 1 " and white pixel is 
marked with "0" in a binary image. The image dilation operation is to expand the 

20 region of black dots by a local neighborhood operation as shown in Figure 3 A. 
The center pixel "o" at the output is set to "0" only when all neighboring pixel Xi 
in a 3x3 window, where 1=1, ... 8, are matched with the 3x3 zero mask. 
Otherwise, the center pixel "o" at the output is marked with black pixel, " 1 ". The 
dilation operation is required because in some documents the text is printed with 

25 dot-matrix. The dilation operation connects the adjacent small dots into line 
segments so that the noise dots and the dots belonging to dot-matrix characters 
becomes distinguishable. A graphical example is illustrated in Figure 3B and 
Figure 3C. Figure 3B displays a binary image before image dilation. The number 
and size of the black dots in the left-handed side of the image is identical to the 

30 one in the right-handed side. If the number (or the area) of the noise dots is taken 
as a feature to evaluate level of noise, the feature will fail to differentiate the 
image patterns in both sides. However, after image dilation is applied, the original 
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dotted pattern is turned into two straight line segments in the left-handed side as 
shown in Figure 3C. The dispersed noise patterns in the right handed side of the 
image stays the same except the enlarged noise dots. Hence, the number of noise 
dots becomes a useful feature in extracting real noise dots in the evaluation of 
5 noise level in the dilated binary image. 

The randomly, distributed small dots which contain no image 
information in a binary image are defined as noise. The noise measurement 
requires counting the number and the area of the small dots in a binary image. 
Figure 3D depicts the processing steps of noise measurement on the dilated binary 

10 image. First, it applies to a contour tracing method (see reference in U.S. Patent 
No. 5,974,199, the disclosure of which is hereby incorporated by reference) to 
extract the outline of every object in the dilated binary image. The contour pixels 
are extracted either in a sequence of clockwise or counterclockwise. The contours 
extracted in the order of clockwise are the outer contour (or black object 

1 5 contours), and the contours extracted in the order of counterclockwise are the 

inner contours (or white object contours). A size threshold (T s ) is applied to reject 
the large contour components whose size is too large to be considered as noise 
dots. The numbers of the remaining small contours are counted for black noise 
dots (NB) and white noise dots (NW), respectively, and the total noise area (NA) 

20 for black noise dots is calculated. The normalized noise area (R) is the ratio of 
NA value over the document area (DA) and is taken as the image noise index 
(INI). The NB and NW values are used to differentiate the white background 
from the black background. When the NB is greater than the NW, it indicates that 
the image has a white background. A large NW value then indicates that the 

25 background is black. This occurs when there is a con-tone image or large dark 
gray area existed in the document. Figure 3E is an example of the calculation of 
noise features. There are 20 black noise dots counted. That is that NB equals to 
20. The number of white noise dots (NW) is counted as 10. Assume that the 
average area of each noise is calculated as Na, then the total noise area (NA) is 20 

30 times Na and the image noise index (INI) is equal to NA/DA. 

The image noise index (INI) is used to judge the noise level of a 
binary image. When the INI value is below a specified threshold (ITH), the image 



is considered as good binary image. That means the binary image is legible and 
the background is clean. When it is greater than the ITH, it is classified as a noise 
binary image. The document is required to rescan or the gray scale image is 
retrieved if available for automatic image quality correction. 
Automatic Image Quality Correction 

Figure 4 shows the block diagram of image processing for 
automatic image quality correction. A multiple adaptive image thresholding 
process is applied to produce (N+M) binary images by orderly change of the N 
different contrast and the M different intensity threshold parameters. The noise 
level of every binary image in the (N+M) thresholded image is calculated, 
respectively. The optimal threshold parameters is obtained from the (N+M) pair 
of threshold values by analyzing the correlation of the (N+M) noise levels and the 
(N+M) pairs of contrast and intensity thresholds. 

The basic processing steps of the multiple adaptive thresholding 
are the same as the one in Figure 2 except with the multiple input of threshold 
values (CT, IT). The input (CT, IT) values are changed with 5 units decrement of 
CT and 30 units increment of IT in order of (60,70), (55,70), (50,70), (45,70), 
(40,70), (35,70), (30,70), (25,70), (20,70), (15,70), (60,100), (60,130), (60,160), 
(60,190) and (60,220). The 15 pairs of threshold values are fed into the adaptive 
thresholding process sequentially and generate 15 different binary images. By 
applying the process of noise feature extraction to the 15 different images 
separately as shown in Figure 3, there are 15 noise values, R(CT,IT), calculated 
and listed in Figure 4B. 

The objective of correlation analysis of noise and (CT,IT) is to 
identify the threshold values (CT,IT) which will be used in the adaptive 
thresholding process in generating least background noises while retaining all 
image information in a thresholded image. Figure 4C is the block diagram of the 
data analysis of selecting the optimal threshold values. First, scanning through the 
set of noise values, R(CT(I),70), where 1=1,. . .,10, if the condition (A): 
R(CT(I),70)>Rth for all CT(I), is met, where the Rth is a predefined noise 
threshold value (Rth) as shown in Figure 4D. This indicates that the noise levels 
of all thresholded images are high and unrelated to the change of contrast 



threshold. Hence, it requires the change of intensity threshold (IT) in search of the 
optimal intensity threshold. On the other hand, if the condition (A) fails, it 
requires the change of the contrast threshold (CT) in search of the optimal contrast 
threshold. 

In the search of optimal contrast threshold, if all of the noise 
values, R(CT(I),70), 1=1, . . .,10, are smaller than the noise threshold (Rth) as 
shown in Figure 4D, it indicates that the starting threshold setting at (CT,IT) = 
(60,70) generates little background noises and the final threshold values 
(CTF,ITF) = (60,70) are taken as the optimal threshold values. Otherwise, by 
examining the array of R(CT(I), 70) data, if the condition (B) : R(CT(I), 
70)>R(CT(I+1), 70), 1-1,. . ., 9, is satisfied, the final threshold values (CTF,ITF) = 
(CT(I), 70) is selected at where the CT(I) meets the condition: R(CT(I), 70)<Rth 
and R(CT(I+1), 70)>Rth. As the example shown in Figure 4E, the CT(I) value is 
selected at 35 and the final threshold values (CTF,ITF) = (35, 70). If the test of 
the condition (B) fails, the final threshold values (CTF,ITF) = (CT(I), 70) is 
selected at where the CT(I) meets the condition: R(CT(I), 70)<R(CT(I-1), 70). As 
the example shown in Figure 4F, the CT(I) value is selected at 40 and the final 
threshold values (CTF,ITF) = (40, 70). 

In the search of optimal intensity threshold, if the condition (C) : 
R(60,IT(J))>Rth for all IT(J), J= 1 , ... ,6, is met, it indicates that the noise levels of 
all thresholded images are high and unrelated to the change of intensity threshold 
as the example shown in Figure 4G. The final threshold values (CTF,ITF) is set at 
(60, 70) which is the starting setup. This implies that there is no optimal threshold 
values detected and the final threshold values are assigned with the starting 
setting, (CTF,ITF) = (60, 70). On the other hand, if the test of the condition ( C ) 
fails, the (CTFJTF) = (60,IT(J)) is selected at where R(60, IT(J-l))>Rth and R(60, 
IT(J))<Rth. As the example shown in Figure 4H, the IT(J) value is selected at 
130. This indicates the intensity threshold at IT(J-l) = 100 is close to the 
background intensity and it is incapable of thresholding the intensity levels of 
background pixels and results in generating large amount of background noises. 
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The invention has been described in detail with particular reference 
to certain preferred embodiments thereof, but it will be understood that variations 
and modifications can be effected within the scope of the invention. 



